The present invention relates to automated language systems. More specifically, the present invention relates to a computer-implemented technique for document editorial quality assessment.
Small, medium and large organizations often have a number of their documents prepared by third parties. Such documents are not always of a consistent quality/style and therefore may have to go through one or more quality/style edits to better enable the intended audience to relatively rapidly and accurately understand the information as it is presented in the documents. Thus, in a typical organization, editors have to review documents received from third parties and, if necessary, send them back for correction. Each iteration of receiving a draft of a document, reviewing it and, if necessary, sending it back for correction is time consuming and costly.
More recently, efforts have been made to develop automatic document quality/style assessment systems. However, the development and/or implementation of such systems has at best only involved partial automation. For example, the development of one such system involves including “hand-coded” style/quality rules in the source code of the system software. These hand-coded rules are contrived by humans (for example, technical writers) who grade a number of documents, for example, and use these grades or scores to construct the rules that form the decision-making foundation of the system. Due to the significant human involvement at the front end of developing the system, cost considerations limit the number of documents that can be graded to develop the rules. This results in compromising the accuracy of the system.
A prior art essay scoring technique is described in an article by Leah S. Larkey and W. Bruce Croft (2003) entitled “A Text Categorization Approach to Automated Essay Grading.” (In: Mark D. Shermis and Jill C. Burstein (eds.) (2003): Automated Essay Scoring. Lawrence Erlbaum Associates.) This technique requires a supervised learning approach. Specifically, the system needs to be trained on essays that have been assigned a score by one or several human graders. Further, this essay scoring technique relies on a relatively small set of features that correlate with grades assigned by the human graders and are often specifically chosen to emulate criteria used by human graders. Additionally, this technique has the goal of assessing writing quality according to an agreed upon standard, shared by the human graders, and defined, for example, in instructions and training of teaching candidates in the educational system who will be responsible for the grading of tests.
There is a need to further automate document editorial quality/style assessment.